Image signal processing apparatus and image signal processing method

ABSTRACT

An image signal processing apparatus may include a partial image data block obtaining unit configured to obtain partial image data blocks by dividing image data corresponding to a predetermined screen having a predetermined number of horizontal and vertical pixels into image data portions, each having a predetermined number of partial horizontal and vertical pixels, at predetermined positions; a packet generating unit configured to generate packets, each storing one of the partial image data blocks; and one or more signal processing units configured to receive the packets from a transmission path through which the image data is transmitted, perform signal processing in units of partial image data blocks stored in the packets, and output the image data in the form of packets to the transmission path.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Japanese Patent Application No. JP2006-211031 filed in the Japanese Patent Office on Aug. 2, 2006, theentire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image signal processing apparatus toperform image signal processing on image data of a predetermined formatcorresponding to image signals, and to an image signal processingmethod.

2. Description of the Related Art

An apparatus to process images is configured to perform one or morepredetermined processes (image signal processing) on image signals inaccordance with the purpose of use.

For example, in an image capturing apparatus to record a captured imageas a moving image or a still image, camera signal processing to performsignal processing to generate image data as a moving image or a stillimage from signals obtained by an image capturing unit (camera) isperformed first, and then other types of image signal processingincluding conversion of an image size and compression/encoding aresequentially performed. Then, the image data to be recorded that iseventually generated by those types of image signal processing iswritten on a medium of a predetermined format and is stored therein.

Patent Document 1 (Japanese Unexamined Patent Application PublicationNo. 2000-92349) describes a configuration related to resolutionconversion performed as image signal processing.

SUMMARY OF THE INVENTION

Image data has a considerably large data amount per unit of reproducingtime compared to voice data, for example. Accordingly, a large-scalecircuit is used to perform image signal processing, and thus aprocessing load applied on a CPU (central processing unit) to controlthe image signal processing is heavy. For these reasons, it ispreferable that an apparatus to perform image signal processing isconfigured to be as efficient as possible with a miniaturized circuit.

In view of the above-described problem, an image signal processingapparatus according to an embodiment of the present invention mayinclude partial image data block obtaining means for obtaining partialimage data blocks by dividing image data corresponding to apredetermined screen having a predetermined number of horizontal andvertical pixels into image data portions, each having a predeterminednumber of partial horizontal and vertical pixels, at predeterminedpositions; packet generating means for generating packets, each storingone of the partial image data blocks; and one or more signal processingmeans for receiving the packets from a transmission path through whichthe image data is transmitted, performing signal processing in units ofpartial image data blocks stored in the packets, and outputting theimage data in the form of packets to the transmission path.

In the above-described configuration, as the image data on which imagesignal processing is to be performed, partial image data blocks eachincluding a predetermined number of partial horizontal and verticalpixels may be formed based on image data of one screen. Herein, thenumber of partial horizontal and vertical pixels may mean the number ofsuccessive horizontal pixels and the number of successive verticalpixels in a part of the total number of horizontal pixels and verticalpixels forming image data of one screen. Then, packets each storing oneof the partial image data blocks may be generated, and the image datamay be input/output to/from the image signal processing means in theform of packets.

In this case, each of the partial image data blocks may be obtained byextracting a rectangle portion of the image data of one screen. The sizethereof may be smaller than that of the image data of the entire screen,and thus each packet has a small size. Accordingly, the partial imagedata block stored in each packet may be used as a minimum processingunit of image data in the signal processing means. Also, the size of anarea to hold image data to be processed by the signal processing meansmay be much smaller compared to a case where image data of an entirescreen is used as an image signal processing unit. This may enable adecrease in hardware scale. Furthermore, the time occupied by signalprocessing of image data of one signal processing unit may besignificantly shortened in accordance with the smaller data size, whichenhances signal processing efficiency.

In the above-described manner, according to an embodiment of the presentinvention, efficiency of image signal processing may be enhanced interms of circuit configuration and processing efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example of a configuration of an imagecapturing apparatus according to an embodiment of the present invention;

FIG. 2 illustrates a processing concept of generating data blocksaccording to the embodiment;

FIG. 3 illustrates the data blocks by type according to an overlappingarea;

FIG. 4 illustrates an example of a configuration of a packet generatedin the image capturing apparatus according to the embodiment;

FIG. 5 illustrates an example of a configuration of a signal processingblock 100 included in the image capturing apparatus according to theembodiment;

FIG. 6 illustrates an example of timings to capture moving and stillimages (image data obtaining timings) during moving/still imagesconcurrent capturing;

FIG. 7 illustrates timings of processing moving and still images duringmoving/still images concurrent capturing according to the embodiment;

FIG. 8 schematically illustrates a procedure of reconstructing packetsaccording to the embodiment;

FIG. 9 illustrates a signal processing block according to amodification;

FIG. 10 illustrates a signal processing block according to anothermodification; and

FIG. 11 illustrates a configuration of an image capturing apparatusaccording to a modification.

DETAILED DESCRIPTION

Hereinafter, an embodiment of the present invention is described. Inthis embodiment, a configuration of an image signal processing apparatusbased on the present invention is applied to an image capturingapparatus capable of capturing and recording moving and still images.

In an image capturing apparatus 1 illustrated in FIG. 1, an imagecapturing unit 11 includes an optical system unit and an imaging deviceunit. The optical system unit includes a lens optical system, a focusmechanism, a shutter mechanism, and an iris mechanism. Light entered theoptical system unit is focused as captured light onto a light receivingsurface of the imaging device unit. The imaging device unit includes aphotoelectric converting device, such as a CCD (charge-coupled device)or a CMOS (complementary metal-oxide semiconductor) sensor, performsphotoelectric conversion on the captured light received by the lightreceiving surface, and outputs electric signals corresponding to asignal charge accumulated in accordance with the captured light receivedby the light receiving surface at predetermined timing.

In this case, a camera signal processing unit 12 performs signalprocessing called preprocessing.

In the preprocessing, the camera signal processing unit 12 receivessignals output from the image capturing unit 11, performs a CDS(correlated double sampling) process and an AGC (automatic gain control)process, and converts the signals to digital image signals through A/D(analog to digital) conversion. Accordingly, image data as a movingimage or a still image can be obtained.

Then, in this embodiment, the image data obtained through theabove-described preprocessing is transferred to an external memory 17via an internal bus 21 and is held therein under control by a controlunit 16. The external memory 17 may be a volatile memory devicerepresented by a DRAM (dynamic random access memory) or an SDRAM(synchronous DRAM) under present circumstances.

Then, in this embodiment, a packet processing unit 13 performspacketization on the image data that is transferred from the camerasignal processing unit 12 and is held in the external memory 17. Thedetails of a method for generating packets by the packet processing unit13 and the structure of the generated packets are described below.Briefly, a two-dimensional image data area corresponding to one screen(one frame in a moving image) is divided into units of partialrectangular area data (data blocks), each including a predeterminednumber of horizontal/vertical pixels, and a header is added to each datablock, so that packets are generated.

Thereafter, the image capturing apparatus 1 according to this embodimentperforms image signal processing while regarding the data block (partialimage data) in each packet generated in the above-described manner as aminimum processing unit. In the configuration illustrated in FIG. 1, afunctional unit performing image signal processing on the packetsincludes a unit performing the preprocessing on digital signals (imagedata) provided in the camera signal processing unit 12, a resolutionconverting unit 14, and an encoding unit 15.

Thus, when the camera signal processing unit 12, the resolutionconverting unit 14, and the encoding unit 15 perform signal processing,data is input thereto in the form of packets from the internal bus 21and predetermined processes are performed on the data blocks stored inthe input packets. After the data blocks have been processed, they arepacketized again and are output to the internal bus 21. With thisconfiguration, in the image capturing apparatus 1 according to thisembodiment, image data to be processed is transmitted in the form ofpackets between the units performing image signal processing.

The camera signal processing unit 12, the resolution converting unit 14,and the encoding unit 15 perform the following types of image signalprocessing on the data blocks in the packets.

First, the preprocessing performed on digital signals (image data) toprocess data blocks by the camera signal processing unit 12 includes aprocess of setting color balance including AWB (auto white balance).

Also, the camera signal processing unit 12 can perform predeterminedsignal processing on signals on which preprocessing for digital signalshave been performed. That is, the camera signal processing unit 12 canperform the preprocessing in a stage of digital signals andpredetermined camera signal processing for image signal data after thepreprocessing. The camera signal processing includes signal processingto apply a special effect, such as conversion to an image of a colorother than full-color, e.g., monochrome or sepia.

The resolution converting unit 14 performs signal processing to convertthe resolution of image data on which the preprocessing and the camerasignal processing have been performed by the camera signal processingunit 12 to another desired resolution. Herein, the resolutioncorresponds to an image size and is expressed by the number ofhorizontal pixels and the number of vertical pixels constituting onescreen.

Typically, the image capturing unit 11 transfers signals obtainedthrough image capturing performed by using all pixels given in theimaging device unit to the camera signal processing unit 12.Accordingly, in the stage of preprocessing in the camera signalprocessing unit 12, signal processing is performed on full-size imagedata corresponding to the number of pixels in the imaging device unit.On the other hand, when a captured moving or still image is to berecorded, an image size different from the above-described full-size isoften set in accordance with an initial set value or user setting. Insuch a case, a process to change the image size is performed. Theresolution converting unit 14 performs the process to change the imagesize by converting the resolution.

The resolution converting unit 14 performs the process to change theimage size (resolution conversion), for example, by performing thinningor interpolation of pixel data forming image data in accordance with apredetermined rule based on a reducing or increasing ratio of the imagesize, so that the resolution according to a desired image size (thenumber of horizontal/vertical pixels) can be obtained.

The encoding unit 15 performs signal processing to compress and encodeimage data as a moving image or a still image by a predetermined method.The compressing and encoding method corresponding to the encoding unit15 is not limited, but may be appropriately changed in accordance withthe format of moving image data or still image data to be recorded orthe format of a storage medium to store a captured image.

A series of image signal processing performed by the camera signalprocessing unit 12, the resolution converting unit 14, and the encodingunit 15 is performed to record a captured image as moving image data orstill image data in a storage medium provided in a storage unit 19.Thus, the image signal processing performed by the camera signalprocessing unit 12, the resolution converting unit 14, and the encodingunit 15 is followed by a process to store the processed image data inthe storage medium.

The image data may be recorded on the storage medium by specifying aunique format based on the form of packets. Alternatively, the imagedata may be recorded in accordance with the form of data to be recordedcorresponding to the format of a standardized storage medium.

In this embodiment, the image data is recorded in accordance with theform of data to be recorded corresponding to the format of thestandardized storage medium, considering general versatility. In thatcase, since the image data is in the form of packets in the image signalprocessing stage, the image data needs to be transformed into the formof recording data corresponding to the format of the storage medium inorder to store the image data in the storage medium.

For this purpose, in this embodiment, unpacketization is performed toextract data blocks and the extracted data blocks are combined together,so that image data of one screen corresponding to the image data beforepacketization is restored. Then, the restored image data to be recordedis transferred to the storage unit 19 and is stored in the storagemedium.

Such restoration of the image data, including unpacketization, isperformed by the packet processing unit 13 by using the external memory17 as a work area under control by the control unit 16. That is, forexample, the packets on which necessary image signal processing has beenperformed are transferred to the external memory 17 and are heldtherein, and then the packet processing unit 13 performs a process torestore the image data on the packets held in the external memory 17.

In the configuration illustrated in FIG. 1, the external memory 17 maybe provided only for packetization and unpacketization of image dataperformed by the packet processing unit 13, but may be used for anotherpurpose. For example, the external memory 17 may be used as a work areato process image data when the camera signal processing unit 12 performspredetermined signal processing, when the resolution converting unit 14performs resolution conversion, or when the encoding unit 15 performscompression/encoding or decoding.

The storage unit 19 to record image data on the storage medium in theabove-described manner includes the storage medium of a predeterminedformat and a drive capable of writing/reading data on/from the storagemedium. The storage medium included in the storage unit 19 is notspecified, but may be a hard disk, a DVD (digital versatile disc), or aflash memory under present circumstances. The storage medium may befixed in the image capturing apparatus together with the drive or may beprovided to the drive in the storage unit 19 in a removable manner. Thestorage unit 19 performs management of stored data corresponding to thestandard of a predetermined file system, so that moving image data orstill image data stored in the storage medium can be managed in units offiles.

The image capturing apparatus 1 also includes a display unit 18. Thedisplay unit 18 includes, for example, a display panel and a displaydriving circuit system of a configuration corresponding to the drivingmethod of the display panel. For example, the camera signal processingunit 12 also performs signal processing to display a captured image on amonitor, and the image data to be displayed obtained by the processingis input to the display unit 18 via the internal bus 21. The displayunit 18 drives the display panel based on the input image data to bedisplayed. Accordingly, the image is displayed on a screen of thedisplay panel of the display unit 18.

Files of the moving image data or still image data recorded on thestorage medium in the storage unit 19 can be read under control by thecontrol unit 16. The data read from the storage medium can be displayedas a reproduced image in the display unit 18. For this purpose, the dataread from the storage medium is processed as necessary. For example, theencoding unit 15 performs demodulation (decompression) with respect tocompression/encoding, or the resolution converting unit 14 converts theresolution. Accordingly, image data to be displayed is obtained and istransferred to the display unit 18.

The control unit 16 controls each unit in the image capturing apparatus1 and actually serves as a microcomputer including a CPU (centralprocessing unit), a RAM (random access memory), and a ROM (read onlymemory) that are mutually connected via a bus. The RAM in the controlunit 16 is mainly used as a work area of the CPU. For example, a programto be executed by the CPU is expanded in the RAM, or an operation resultof the CPU is temporarily stored in the RAM. The ROM stores a program tobe executed by the CPU and setting data required in various processes.

An operation unit 20 serves as an input operation unit including variousoperation buttons provided at a predetermined position of the casing ofthe image capturing apparatus 1 and a touch panel. The operation unit 20outputs an operation information signal according to the result of anoperation performed to the control unit 16 (CPU). The control unit 16performs control and processing in accordance with the input operationinformation signal.

As can be understood by the description with reference to FIG. 1, in theimage capturing apparatus 1 according to this embodiment, image data ofone screen is divided and packetized, and the packetized image data istransmitted among the image signal processing units including the camerasignal processing unit 12, the resolution converting unit 14, and theencoding unit 15. These image signal processing units performpredetermined types of image signal processing while regarding the datablock (partial image data) extracted from each input packet as aprocessing unit.

In this embodiment, image signal processing is performed withtransmission and input/output in units of packets, so that efficiency ofthe image signal processing is enhanced. This is described in detailbelow.

First, packetization of image data performed by the packet processingunit 13 is described.

FIG. 2 schematically illustrates image data of one screen by means of anarray of pixel data segments along the horizontal and verticaldirections. Herein, the image data of one screen means image data of oneframe of a moving image or image data of one still image. In the imagedata of one screen, the number of horizontal pixels is represented by“p” and the number of vertical pixels is represented by “q”. That is,pixel data array of p×q constitutes the image data of one screen.

In this embodiment, such image data of one screen is packetized by thepacket processing unit 13 and is held in the external memory 17.

Before packetization is performed by the packet processing unit 13, theimage data illustrated in FIG. 2 is divided into a predetermined numberof image data portions, each including a predetermined number of partialhorizontal pixels and partial vertical pixels. The image data portionsgenerated accordingly are regarded as data blocks.

More specifically, as illustrated in FIG. 2, each square image dataportion including the number of horizontal pixels N/the number ofvertical pixels N corresponds to the data block. In this case, thescreen is divided not equally based on the number of horizontal pixelsN/the number of vertical pixels N, but is divided so that an overlappingarea of the number of pixels “a” is generated between data blocksadjoining in the horizontal and vertical directions (overlappingdivision), as illustrated in FIG. 2. As a result of the overlappingdivision, n×m data blocks: n columns from column C₁ to column C_(n) andm lines from line L₁ to line L_(m), can be generated in the image dataof one screen.

The data blocks generated in the above-described manner can beclassified into nine types in accordance with a placement state of theoverlapping area ARo defined by the number of pixels “a” in the entirescreen. For reference, the nine types of data blocks are illustrated in(a) to (i) of FIG. 3. Herein, the positions of the respective datablocks on the screen are indicated by coordinates, which are expressedby the column numbers C₁ to C_(n) and the line numbers L₁ to L_(m)corresponding to the respective data blocks illustrated in FIG. 2.

First, in (a) of FIG. 3, the data block at the coordinates (C₁, L₁) inthe uppermost line and the left-end column is shown. In the data blockat the coordinates (C₁, L₁), the pixels along the upper side and theleft side thereof correspond to the pixels at an end portion of theoriginal screen. There is no adjoining data block at the upper side andthe left side, and thus there is no overlapping area. On the other hand,there are adjoining data blocks on the right side and the lower side,and thus the overlapping area ARo where the number of pixels “a”overlaps exists between the adjoining data blocks.

In (b) of FIG. 3, the data blocks at the coordinates (C₂, L₁) to(C_(n-1), L₁), other than the data blocks at the right-end and left-endcolumns in the uppermost line of the screen, are shown. In these datablocks, the overlapping area ARo exists on the right, left, and lowersides.

In (c) of FIG. 3, the data block at the coordinates (C_(n), L₁) in theuppermost line and the right-end column of the screen is shown. In thisdata block, the overlapping area ARo exists on the left and lower sides.

In (d) of FIG. 3, the data blocks at the coordinates (C₁, L₂) to (C₁,L_(m-1)), other than the data blocks in the uppermost and lowermostlines in the left-end column of the screen, are shown. In these datablocks, the overlapping area ARo exists on the upper; lower, and rightsides.

In (e) of FIG. 3, the inner data blocks at the coordinates (C₂, L₂) to(C_(n-1), L_(m-1)), other than the data blocks on outer edges, that is,the left- and right-end columns and the uppermost and lowermost lines ofthe screen, are shown. In these data blocks, the overlapping area ARoexists on all of four sides: the right, left, upper, and lower sides.

In (f) of FIG. 3, the data blocks at the coordinates (C_(n), L₂) to(C_(n), L_(m-1)), other than the data blocks in the uppermost lowermostlines in the right-end column of the screen, are shown. In these datablocks, the overlapping area ARo exists on the upper, lower, and leftsides.

In (g) of FIG. 3, the data block at the coordinates (C₁, L_(m)) in thelowermost line and the left-end column of the screen is shown. In thisdata block, the overlapping area ARo exists on the right and uppersides.

In (h) of FIG. 3, the data blocks at the coordinates (C₂, L_(m)) to(C_(n-1), L_(m)), other than the data blocks in the right- and left-endcolumns in the lowermost line of the screen, are shown. In these datablocks, the overlapping area ARo exists on the right, left, and uppersides.

In (i) of FIG. 3, the data block at the coordinates (C_(n), L_(m)) inthe lowermost line and the right-end column of the screen is shown. Inthis data block, the overlapping area ARo exists on the left and uppersides.

The overlapping area ARo is provided for the following reason.

For example, in some image signal processing including interpolation ofpixels and compressing/encoding, pixel data at a reference position andpixel data at a predetermined distance from the reference position maybe used to perform a process on a pixel position. In this embodiment,image signal processing is performed while regarding the data blockstored in each packet as a processing unit, as described below. In thiscase, if the data blocks are formed by simply dividing the image data ofone screen such that no overlapping area is provided and if signalprocessing is performed by using a pixel positioned on the periphery ofthe data blocks as a reference, an appropriate signal processing resultis not obtained because of the absence of necessary neighboring pixels.In order to avoid such inconvenience, the overlapping area ARo wherepixels are overlapped in the horizontal and vertical directions isprovided.

Therefore, the number of pixels serving as the overlapping area ARo maybe set in accordance with a distance to neighboring pixel data that isnecessary in actual image signal processing. As described above withreference to FIG. 2, in the overlapping area ARo, the portion along thehorizontal direction and the portion along the vertical direction have awidth of the same number of pixels “a”. However, the portions along thehorizontal and vertical directions may have different widths each havinga different number of pixels, depending on signal processing.

In the above-described manner, the packet processing unit 13 generatesdata blocks by defining image data portions in the image data of onescreen in accordance with a predetermined rule. Then, the packetprocessing unit 13 regards the generated data blocks as operands andadds a header to each of them, so as to generate packets.

FIG. 4 illustrates an example of a configuration of a packet generatedby the packet processing unit 13.

As illustrated in (a) of FIG. 4, the entire packet includes a header anda data block that follows the header.

In FIG. 4, (b) illustrates an example of a configuration of the header.In this example, the header includes area coordinates, area size, frameID, command flag, processing path information, history information, andextended information.

The area coordinates are information indicating the position on thescreen of a partial data area as the data block stored as an operand inthe packet. For example, information of the horizontal and verticalcoordinates of pixel data on the screen can be used as the areacoordinates.

That is, in view of the correspondence with FIG. 2, where the image dataof one screen is illustrated with the number of horizontal pixels p andthe number of vertical pixels q, the position of each piece of pixeldata forming the screen can be indicated by coordinates. For example,the pixel data in the uppermost line and the left-end column can beindicated by pixel coordinates (1, 1), the pixel data in the uppermostline and the right-end column can be indicated by pixel coordinates (p,1), the pixel data in the lowermost line and the left-end column can beindicated by pixel coordinates (1, q), and the pixel data in thelowermost line and the right-end column can be indicated by pixelcoordinates (p, q). Accordingly, the pixel coordinates of the pixel dataat a specific position in the pixel data forming the position of a datablock can be used as the area coordinates. In this embodiment, the pixelcoordinates of the pixel data at the upper left corner, that is, in theuppermost line and the left-end column in the data block, are stored asinformation of the area coordinates.

In the pixel coordinates of the pixel data at the upper left corner,that is, in the uppermost line and the left-end column in the datablock, the horizontal and vertical coordinates can be indicated in thefollowing way.

The number of horizontal pixels and the number of vertical pixelsincluded in a block data are represented by “N”, respectively, thenumber of overlapping pixels in the overlapping area ARo is representedby “a”, the column number of each of the columns arranged from left toright in the horizontal direction in a case where the data blocks arearranged on one screen as illustrated in FIG. 2 is represented by “n”(natural number), and the line number of each of the lines arranged fromthe top to bottom in the vertical direction is represented by “m”(natural number). In this case, the horizontal coordinate can beexpressed by the following expression (1).

1+(N−a)(n−1)  (1)

Also, the vertical coordinate can be expressed by the followingexpression (2).

1+(N−a)(m−1)  (2)

For example, the packet processing unit 13 manages the positions of thedata blocks by using the above-described line numbers “m” and columnnumbers “n” in a process of dividing the image data of one screen intodata blocks, so that the packet processing unit 13 can obtain the valueof pixel coordinates (horizontal coordinate, vertical coordinate) to bestored in the field of the area coordinates in the header added to thedata block by using expressions (1) and (2).

In the configuration of the header illustrated in (b) of FIG. 4, thearea size is information indicating the size of the area of the datablock. In this embodiment, information of the number of horizontalpixels×the number of vertical pixels (=N×N) is stored as the area size.

The frame ID is identification information to specify the frame numberof original frame image data in a case where the image data of the onescreen (frame image data) forms a moving image. When the image data ofthe screen is a still image, a predetermined special value that is notused for a moving image is stored as the frame ID. Thus, by referring tothe frame ID, it can be determined whether the data block in the packetbelongs to a moving image or a still image.

As described above with reference to FIG. 1, the packets on which allnecessary image signal processing have been performed are restored intoimage data corresponding to the original image of one screen by thepacket processing unit 13. At that time, the packet processing unit 13refers to at least the area coordinates and the frame ID stored in theheader as position information indicating the position of the data blockin the image data. Accordingly, the frame to which the data block storedin the packet belongs and the position in the frame are specified. Then,the packet processing unit 13 collects the data blocks belonging to thesame frame image (still image) and places the data blocks in the correctcoordinate positions. Accordingly, the image data is appropriatelyrestored.

The command flag (parameter specifying information) is information tospecify various parameters used in image signal processing. For example,such specifying information includes image size specifying informationused in conversion of resolution performed by the resolution convertingunit 14, and selection of a compressing and encoding method andinformation specifying a compression ratio used in a compressing andencoding process performed by the encoding unit 15. These various typesof specifying information are stored in the field of the command flag ina predetermined configuration.

The processing path information (signal processing specifyinginformation) is information to specify a path of signal processing to beperformed on the data block stored in the packet.

The configuration of the processing path information includes four bits,as illustrated in (c) of FIG. 4. Preprocessing, camera signalprocessing, resolution conversion, and encoding are assigned to the fourbits from the most significant bit to the least significant bit. Herein,the preprocessing means preprocessing performed on the data block in thepacket by the camera signal processing unit 12. The camera signalprocessing means signal processing performed after the preprocessing bythe camera signal processing unit 12. The resolution conversion and theencoding mean the processes performed by the resolution converting unit14 and the encoding unit 15, respectively.

The value stored in each bit position corresponding to thepreprocessing, camera signal processing, resolution conversion, andencoding is defined as follows. For example, “1” indicates that theprocess corresponding to the bit position is to be performed, while “0”indicates that the process corresponding to the bit position is skipped.Specifically, when the processing path information indicates 1101 fromthe most significant bit to the least significant bit, preprocessing,camera signal processing, and encoding are performed on the data blockin the packet, but resolution conversion is skipped. Note that theparameters set in each of the preprocessing, camera signal processing,resolution conversion, and encoding are specified based on theinformation stored in the command flag.

The history information (progress indicating information) is informationindicating the result of signal processing performed on the data blockstored in the packet. An example of the configuration thereof isillustrated in (d) of FIG. 4. As in (c) of FIG. 4, the configurationincludes four bits to which preprocessing, camera signal processing,resolution conversion, and encoding are assigned from the mostsignificant bit to the least significant bit. The value stored in eachbit position is defined as follows. That is, “1” indicates that theprocess has been performed, while “0” indicates that the process has notbeen performed. For example, when the history information indicates1110, preprocessing, camera signal processing, and resolution conversionhave been performed, but encoding has not been performed.

The extended information (processing result information) is varioustypes of additional information about the data block stored in thepacket. Some of the information about the image data can be obtainedonly after some processing has been performed on the image data. Forexample, an evaluation value of complexity of an image (frequencyinformation) is obtained as one of results of signal processing in apredetermined stage. Such information is included in the extendedinformation.

The configuration of the header of the packet illustrated in FIG. 4 isonly an example and can be appropriately changed in accordance with theconfiguration of actual signal processing.

Hereinafter, an example of signal processing performed on the packetsgenerated in the above-described manner is described with reference toFIG. 5.

FIG. 5 illustrates an example of a configuration of a signal processingblock 100. The signal processing block 100 corresponds to a common parthaving a function of performing image signal processing on input/outputpackets of the camera signal processing unit 12, the resolutionconverting unit 14, and the encoding unit 15 in the image capturingapparatus 1 according to this embodiment.

In the signal processing block 100, image data to be processed isinput/output in the form of packets having the configuration illustratedin FIG. 4. Each of the packets input to the signal processing block 100is input to a header analyzing unit 101 first.

The header analyzing unit 101 unpacketizes the packet input from theinternal bus 21 so as to separately obtain a header and a data block,and analyzes the header. That is, the header analyzing unit 101recognizes the content of the header and sets an operation mode ofsignal processing in the signal processing block 100 in accordance withthe recognition result. Herein, setting of an operation mode includessetting of whether signal processing is to be performed and setting ofparameters used in each type of signal processing. The header analyzingunit 101 sets an operation of a sequencer 104 so that the setting resultof the operation mode is reflected thereon. The content of the headerseparated by the header analyzing unit 101 is transferred to a headeradding unit 108.

If the data block in the packet is to be processed by the signalprocessing block 100 as a result of the analysis made by the headeranalyzing unit 101, the data block separated from the header by theheader analyzing unit 101 is transferred to a line buffer 102 withpredetermined timing. The line buffer 102 outputs sequentiallytransferred data blocks in units of horizontal lines. Due to such inputand output performed by the line buffer 102, each data block having aconfiguration of an operand of a packet can be dealt as being restoredto a partial image data position having a two-dimensional configurationillustrated in FIGS. 2 and 3 in the subsequent stage of the line buffer102.

In the subsequent stage of the line buffer 102, a predetermined numberof image signal processing units: a first processing unit 103-1 to ann-th processing unit 103-n, are provided. The number of image signalprocessing units to be provided and the signal processing performed byeach of the provided image signal processing units are appropriatelychanged depending on the actual configuration of the signal processingpart as the signal processing block 100. In FIG. 5, the first processingunit 103-1 to the n-th processing unit 103-n serving as the image signalprocessing units are mutually connected in series. Alternatively, theseimage signal processing units may be mutually connected in parallel.Also, a series connection and a parallel connection can be usedtogether.

Registers 105-1 to 105-n are provided for the first processing unit103-1 to the n-th processing unit 103-n, respectively.

The sequencer 104 sets appropriate parameters to the respectiveregisters 105-1 to 105-n in accordance with the setting made by theheader analyzing unit 101. The first processing unit 103-1 to the n-thprocessing unit 103-n perform appropriate signal processing operationsbased on the parameters set in the corresponding registers. Herein, thesignal processing operations include an operation of skippingsubstantial signal processing. In this way, the data on which signalprocessing has been performed by the first processing unit 103-1 to then-th processing unit 103-n is output to the header adding unit 108.

The header adding unit 108 receives the data of the header transferredfrom the header analyzing unit 101 and the image data (originally datablock) processed by the image signal processing units. In the headeradding unit 108, predetermined information in the header is rewritten asnecessary on the basis of the signal processing result of the imagesignal processing units. Then, the header adding unit 108 converts theimage data processed by the image signal processing units to a blockdata configuration as an operand of the packet and adds the headerthereto so as to perform packetization. The packet generated in this wayis output via the internal bus 21.

As an example, an operation of the signal processing block 100 in theresolution converting unit 14 is described. In this case, the headeranalyzing unit 101 determines whether resolution conversion is to beperformed by referring to the processing path information in the header.If a value “0” is stored in the bit position of the resolutionconversion, that is, if the resolution conversion should be skipped,this packet is output from the header adding unit 108 to the internalbus 21. In some cases, the packet may be abandoned and is not output tothe internal bus 21.

On the other hand, if a value “1” is stored in the bit position of theresolution conversion, the resolution conversion should be performed. Inthat case, the bit position of the resolution conversion in the historyinformation is further referred to and it is determined whether the bitvalue is “1” (process has been performed) or “0” (process has not beenperformed).

If the bit value indicates that the process has been performed, thesignal processing block 100 need not perform the signal processingagain. Accordingly, as in the case where “0” is stored in the bitposition of the resolution conversion in the processing pathinformation, the packet is output to the internal bus 21 or abandoned.On the other hand, if the process has not been performed, the headeranalyzing unit 101 transfers the data block to the line buffer 102 forresolution conversion. In this case, the header analyzing unit 101recognizes instructions about the resolution conversion stored in thecommand flag and sets parameters in the sequencer 104 in accordance withthe recognition result. The parameters used here include the ratio of aconverted image size to a reference image size. In accordance with theset parameters, the image signal processing system (first processingunit 103-1 to n-th processing unit 103-n) performs thinning orinterpolation of pixels for resolution conversion on the data block.

The header adding unit 108 receives the data block on which resolutionconversion has been performed from the image signal processing system inthe above-described manner. Also, the header adding unit 108 rewritesthe data of the header received from the header analyzing unit 101, forexample, sets “1” indicating that the process has been performed in thebit position of the resolution conversion in the history information.Then, the header adding unit 108 adds the rewritten header to the datablock on which resolution conversion has been performed so as toregenerate the packet and outputs it.

Also, an example of a usage pattern of the extended information in theheader is described.

As described above, the extended information is information that isobtained through signal processing performed on the data block andincludes an evaluation value of complexity of an image. The evaluationvalue of complexity of the image can be obtained based on a frequencycharacteristic of an image signal as a data block. For example, thecontent of an image becomes more complex as the level of ahigh-frequency band becomes higher as the frequency characteristic.Accordingly, the evaluation value of complexity of the image can becalculated based on the frequency characteristic of the data block.

Such a frequency characteristic of an image signal can be actuallydetected in a predetermined processing stage in the camera signalprocessing unit 12. The camera signal processing unit 12 calculates theabove-described evaluation value based on the detected frequencycharacteristic during processing of the data block. This evaluationvalue is stored as extended information in the header when thesignal-processed data is output as a packet from the camera signalprocessing unit 12.

This evaluation value can be used when the encoding unit 15 performscompression/encoding. That is, the signal processing block 100 in theencoding unit 15 performs compression/encoding by changing and setting acompression ratio (encoding efficiency) in accordance with theevaluation value (complexity of the image) recognized by headeranalysis. Accordingly, compression/encoding is performed based on thecompression ratio that is appropriate for the content of the image.

As can be understood from the above-described configuration andoperation of the signal processing block 100, the signal processingblock 100 uses a data block stored in each packet, that is, partialimage data having a size of the number of partial horizontal pixelsN×the number of partial vertical pixels N, as a processing unit. Withthis method, the size of a buffer memory to temporarily hold image datacan be significantly reduced compared to a configuration of typicalimage signal processing where image data of one screen is used as aprocessing unit. As such a buffer memory, the line buffer 102illustrated in FIG. 5 is used. If image data of one screen is used as aprocessing unit, a line buffer having the size corresponding to thenumber of horizontal pixels in one screen is required. On the otherhand, in this embodiment, the size corresponding to a smaller number ofpixels N may be used. Although not illustrated in FIG. 5, in a casewhere a buffer memory to hold image data of a processing unit isrequired, the size corresponding to one frame is unnecessary, but thesize corresponding to a data block is used in this embodiment.

Typically, each signal processing unit can accept a next process onlyafter a process of data of a predetermined processing unit hascompleted. In a case where image data of one screen is regarded as aprocessing unit, if a process of data of one screen has not ended in asignal processing unit, the data is not transferred to the signalprocessing unit in the subsequent stage and the end of the process iswaited for even if signal processing has ended in the previous stage.That is, processing timing is adjusted and thus processing efficiency islikely to decrease. On the other hand, in this embodiment where the datasize of a processing unit is small, time occupied to process eachprocessing unit in each signal processing unit can be significantlyshortened. Accordingly, the above-described factors decreasing theprocessing efficiency can be significantly reduced.

According to the above description, the signal processing block 100determines the necessity of signal processing based on the informationstored in the header of each packet. When signal processing is to beperformed, the signal processing block 100 determines and sets theprocessing based on predetermined information stored in the header andthen performs the signal processing. This means that the signalprocessing block 100 is capable of independently performing anappropriate signal processing operation by using the information storedin the header. That is, in the image capturing apparatus 1 according tothis embodiment, part of a signal processing function of the camerasignal processing unit 12, the resolution converting unit 14, and theencoding unit 15 are capable of independently performing signalprocessing.

Typically, setting of whether signal processing is to be performed andsetting of parameters in those signal processing units are performed bythe CPU. Compared to this configuration, the CPU in the control unit 16according to this embodiment need not perform a processing step tocontrol the signal processing units, and thus a processing load isreduced accordingly.

As described above, in this embodiment, efficiency of image signalprocessing is enhanced in terms of hardware resources and processingefficiency.

A typical image capturing apparatus such as a video camera apparatusunder present circumstances is capable of capturing and recording astill image as well a normal moving image. During capturing andrecording of a still image, a still image can be captured by setting ashooting mode for a still image and by performing an operationequivalent to a press of a shutter button, as in a normal still camera.Also, an apparatus having a function capable of capturing and recordinga still image by performing an operation equivalent to a press on ashutter button during capturing and recording of a moving image(hereinafter referred to as moving/still images concurrent capturing)has been known.

The image capturing apparatus 1 according to this embodiment is alsocapable of capturing and recording a still image in addition to a movingimage and is further capable of performing the above-describedmoving/still images concurrent capturing. Hereinafter, an example ofsignal processing performed during the moving/still images concurrentcapturing by the image capturing apparatus 1 is described.

FIG. 6 schematically illustrates an example of capturing timings ofmoving and still images when the image capturing apparatus 1 accordingto this embodiment is performing the moving/still images concurrentcapturing.

Referring to FIG. 6, assume that a user performs an operation to startcapturing and recording of a moving image at time t0. In accordance withthis operation, the image capturing apparatus 1 continues capturing andrecording of the moving image from time t0 to the end of capturing ofthe moving image. During the capturing and recording of the movingimage, the image capturing apparatus 1 converts image signals obtainedthrough capturing to image data by performing preprocessing of apredetermined stage in the manner described above. Then, the imagecapturing apparatus 1 packetizes the image data and appropriatelyperforms predetermined image signal processing in units of packetsthereafter. Then, the image capturing apparatus 1 sequentially writesthe image data on which the image signal processing has been performedas moving image data to be recorded on the storage medium so that themoving image data is stored therein.

In this case, a shutter operation to capture a still image is performedat time t1, time t2, time t3, and time t4 during recording of the movingimage after time t0. In accordance with the shutter operations tocapture still images, images captured at the shutter operation timingsare obtained as still image data segments Pic1, Pic2, Pic3, and Pic4,each corresponding to one still image, as illustrated in FIG. 6. Forexample, each of the still image data segments Pic1, Pic2, Pic3, andPic4 can be obtained by copying image data of the frame corresponding tothe shutter operation timing from the image data in units of frames thatcan be obtained by the preprocessing performed by the camera signalprocessing unit 12.

As described above with reference to FIG. 6, image data of a still imagecan be obtained in parallel with image data of a moving image inaccordance with the operation of moving/still images concurrentcapturing. The image capturing apparatus 1 according to this embodimentappropriately records the image data of the moving and still imagesobtained in the above-described manner in the storage medium. For thispurpose, adequate image signal processing including preprocessing,camera signal processing, resolution conversion, andcompression/encoding needs to be performed on the image data of themoving and still images. The image capturing apparatus 1 according tothis embodiment performs the signal processing on the image data of themoving and still images in the following way.

The signal processing described above with reference to FIGS. 2 to 6 isperformed on the image data of the moving image. That is, the image datais converted to digital image data in the camera signal processing unit12, the digital image data is divided into data blocks so as to bepacketized, and the signal processing thereafter is performed byinputting/outputting (transferring) the data to the signal processingunits in units of packets.

Also, in the image data of the still images, image data of one stillimage (in FIG. 6, each of the still image data segments Pic1, Pic2,Pic3, and Pic4) is divided into data blocks so as to be packetized.

The processing to packetization may be performed as processing for thestill images independently from processing for the moving image, butpacketization on the frame image during the above-described processingon a moving image can be used. That is, in the moving/still imagesconcurrent capturing, image data of a still image captured at a shutteroperation timing during capturing and recording of a moving image isextracted. During capturing and recording of a moving image, a frameimage captured at a shutter operation timing can be used as image dataof a still image. In a processing sequence of a moving image, the frameimage captured at the above-described shutter operation timing is alsodivided into blocks and is packetized. Thus, by copying packets of theimage data of one frame obtained in this way for a still image,packetized still image data of one screen can be obtained.

As described above, in this embodiment, image data of both moving andstill images are packetized in the moving/still images concurrentcapturing. Then, signal processing is performed at the timingsillustrated in FIG. 7 on the packets obtained in this way.

In FIG. 7, (a) and (b) illustrate processing timings for respectivepackets of moving image data and still image data. The processingtimings illustrated in this figure are the timings of signal processingperformed by the signal processing block 100 in one of the image signalprocessing units (camera signal processing unit 12, resolutionconverting unit 14, and encoding unit 15).

First, the processing timings of the moving image data are described. Attime t0 in (a) of FIG. 7, sequential input of packets starts.Accordingly, the signal processing block 100 sequentially andcontinuously processes the input packets. At time t1, input of thepackets of image data stops, and thus the signal processing block 100stops the processing on the packets accordingly. At time t2 after sometime from time t1, input of packets of the moving image starts again,and thus processing on the packets starts accordingly. That is, a movingimage processing period where a predetermined number of packets aresequentially processed and a pause of moving image processing whereprocessing of packets is paused alternately appear. The pause of movingimage processing corresponds to, for example, a vertical blanking periodof image signals.

On the other hand, signal processing performed on still image dataaccording to this embodiment is illustrated in (b) of FIG. 7. That is,the still image data is sequentially processed in units of packets byusing the pause of moving image processing, where no signal processingis performed on the moving image data.

For example, conventional processing of still image data during themoving/still images concurrent capturing is as follows.

In accordance with a shutter operation during capturing and recording ofa moving image, image data of an image captured by this operation isextracted and is held in an internal buffer memory. After the capturingand recording of the moving image have ended and after the correspondingmoving image data has been stored in a storage medium, signal processingincluding camera signal processing, resolution conversion, andcompression/encoding is performed on the still image data that has beenheld in the internal buffer memory and the still image data is stored inthe storage medium.

In the processing of the still image data, the period when the imagecapturing apparatus performs an operation for recording data aftercapturing a moving image is relatively long. Such an operation isuncomfortable for a user. Also, the image capturing apparatus does notrespond to an operation (e.g., reproducing of a recorded file)immediately after capturing and recording of a moving image, whichcauses stress.

On the other hand, in this embodiment, processing of still image datacan be intermittently performed in accordance with progress ofprocessing of moving image data at the signal processing timingsillustrated in FIG. 7. Accordingly, processing of still image data canbe completed during capturing and recording of a moving image. Also,even if processing of still image data does not complete, the processingcan be completed to some extent during capturing and recording of amoving image, and thus the processing time after capturing and recordingof the moving image can be shortened. That is, the above-describedinconvenience is solved or alleviated.

According to the illustration in FIG. 7, a packet of still image data isprocessed during a pause of moving image processing. This is for clearlyillustrating that the pause of moving image processing is the processingtiming for a packet of still image data. In practice, two or morepackets of still image data may be processed in accordance with therelationship between the time length actually obtained as the pause ofmoving image processing and the time length required to process a packetthat depends on actual signal processing. Furthermore, when as manypackets as possible are processed during the pause of moving imageprocessing, the speed of processing the still image data can beincreased, so that processing efficiency enhances.

In this embodiment, while signal processing of image data progresses,the size of data block stored in a packet may become different from thesize at the packetization. For example, this is when resolutionconversion is performed by the resolution converting unit 14. In thiscase, the number of pixels (the number of pieces of pixel data) forminga data block at packetization is converted, and thus the size of thedata block after the processing is changed in accordance with the numberof pixels after conversion.

For example, when the number of pixels is increased in the resolutionconversion, the size of the data block stored in the packet alsoincreases. In this case, for example, the signal processing unit toreceive and process the packet after resolution conversion may beincapable of accepting the data block if the increased size of the datablock exceeds the size that can be processed. This causes aninconvenience that a signal processing error occurs.

In this embodiment, in order to avoid such inconvenience, a packet isreconstructed in the signal processing block 100 in the resolutionconverting unit 14, as illustrated in FIG. 8.

In FIG. 8, (a) illustrates a packet storing a data block of which sizeis larger than that before resolution conversion, for example. In orderto reconstruct this packet, the original data block is divided into n−1parts, each having a predetermined size, so as to obtain split datablocks #0 to #n, as shown in the transition from (a) to (b) of FIG. 8.Then, headers #0 to #n are added to the split data blocks #0 to #n,respectively, so that n−1 packets are generated. At this time, theheaders #0 to #n are regenerated by rewriting necessary informationitems based on the content of the original header illustrated in (a) ofFIG. 8. For example, since the data block is divided, area coordinatesand area size change with respect to the original block data (the areacoordinates of the split data block # are the same as those of theoriginal data block). Thus, those pieces of information are rewritten inaccordance with the corresponding split data blocks.

Such a process of reconstructing packets may be performed by the headeradding unit 108 illustrated in FIG. 5, for example. In the procedureillustrated in FIG. 8, the packet illustrated in (a) is generated andthen the packet is divided as illustrated in (b) for reconstruction.However, this illustration is for clear understanding, and the packetillustrated in (a) need not be temporarily formed in an actual processof reconstructing packets by the header adding unit 108. That is, splitdata blocks and corresponding headers are generated by using theoriginal packet transferred from the header analyzing unit 101 and theprocessed data block output from the first to n-th processing units, andthen packets are regenerated accordingly.

Hereinafter, modifications of the configuration according to theabove-described embodiment are described with reference to FIGS. 9 to11.

FIG. 9 illustrates a modification of the signal processing block 100. InFIG. 9, parts that are the same as those in FIG. 5 are denoted by thesame reference numerals and the corresponding description is omitted.

In the signal processing block 100 illustrated in FIG. 9, adecompressing unit 109 and a compressing unit 110 are provided. Thisconfiguration is made so that, when a packet is transmitted through thetransmission path (internal bus 21), the data block stored therein iscompressed. Although not illustrated in the drawings, this configurationis realized based on the assumption that the packet processing unit 13performs packetization by compressing a data block in a process ofgenerating the data block.

For example, upon input of a packet to be processed to the signalprocessing block 100 illustrates in FIG. 9, the header analyzing unit101 separates the data block from the header and outputs the data blockto the decompressing unit 109. The decompressing unit 109 receives thedata block and performs decompression (decoding) corresponding to thecompression method of the data block. Then, the data block that has beenrestored to the original format before compression by decompression isoutput to the line buffer 102, so that signal processing is performedthereon. The compressing unit 110 receives the data block that has beenprocessed by the first to n-th processing units 103-1 to 103-n andperforms compression thereon in accordance with a predetermined method.Then, the compressing unit 110 outputs the compressed data block to theheader adding unit 108. The header adding unit 108 adds the header tothe compressed data block and outputs it to the internal bus 21(transmission path).

With this configuration of the signal processing block 100, each packettransmitted between the signal processing blocks can maintain a statewhere the data block therein is compressed. By transmitting the datablock while being compressed, the amount of traffic in the transmissionpath, such as the internal bus 21, is suppressed and transmissionefficiency enhances.

FIG. 10 illustrates another modification of the signal processing block100. In FIG. 10, parts that are the same as those in FIG. 5 are denotedby the same reference numerals and the corresponding description isomitted.

The signal processing block 100 illustrated in FIG. 10 is configured sothat the data block is encrypted while the packet is transmitted throughthe transmission path. For this purpose, a decrypting unit 111 and anencrypting unit 112 are provided.

For example, upon input of a packet to be processed to the signalprocessing block 100 illustrated in FIG. 10, the data block separatedfrom the header in the header analyzing unit 101 is input to thedecrypting unit 111. The decrypting unit 111 decrypts the input datablock. Then, the decrypted data block is input to the signal processingsystem (line buffer 102 and first to n-th processing units 103-1 to103-n), where predetermined processing is performed. Then, theencrypting unit 112 encrypts the data block output from the signalprocessing system and outputs the data block to the header adding unit108. The header adding unit 108 adds the header to the encrypted datablock and outputs it to the internal bus 21 (transmission path).

With this configuration of the signal processing block 100, each packettransmitted between the signal processing blocks can maintain a statewhere the data block therein is encrypted. For example, even when thesignal processing units connected via a transmission path are configuredas black boxes, if a packet is being transmitted through thetransmission path, the content of the packet can be observed. In such acase, that is, in a case where security and confidentiality are requiredin signal processing of a captured image, the configuration illustratedin FIG. 10 enables increased security of the content of image data to beprocessed.

The configuration adopting compression/decompression orencryption/decryption illustrated in FIGS. 9 and 10 can be used forimage data of one screen or data of a larger unit, as well as units ofpackets (data blocks). However, by performing packetization so as toprocess data in units of data blocks as in this embodiment, the circuitscale for compression/decompression or encryption/decryption issignificantly decreased, which is advantageous in terms of cost and acircuit mounting size.

Hereinafter, another modification is described with reference to FIG.11. The configuration illustrated in FIG. 11 is a modification of theconfiguration illustrated in FIG. 1. In FIG. 11, parts that are the sameas those in FIG. 1 are denoted by the same reference numerals and thecorresponding description is omitted.

In this configuration, an internal memory 22 is provided. The internalmemory 22 includes an SDRAM (synchronous dynamic random access memory)or an SRAM (static RAM) of a predetermined size, and connects to thecamera signal processing unit 12, the resolution converting unit 14, andthe encoding unit 15 via a data bus as illustrated in FIG. 11. That is,the internal memory 22 connects to the signal processing unitsperforming signal processing on packets.

In this modification, packets are transmitted among the camera signalprocessing unit 12, the resolution converting unit 14, and the encodingunit 15 through writing/reading on/from the internal memory 22, not viathe internal bus 21. In accordance with this configuration in whichpackets are transmitted not via the internal bus, the packet processingunit 13 is provided in the camera signal processing unit 12. That is,the internal memory 22 and writing/reading thereon/therefrom constitutethe transmission path of packets.

With this configuration, the communication path used by the control unit16 (CPU) to control the other part and the path to transmit packets areindependent from each other. Accordingly, the amount of traffic can bereduced in each of communication in a control system on the control unit16 side and communication for transmitting packets, so that efficiencycan be enhanced.

The present invention is not limited to the configurations according tothe above-described embodiment and modifications, but can beappropriately modified. For example, a video camera is used as the imagecapturing apparatus in the above-described embodiment, but another typeof image capturing apparatus, such as a digital still camera, can beused. In the above-described embodiment, the image signal processingperformed on packets includes part of camera signal processing,resolution conversion, and encoding, in accordance with theconfiguration of the image capturing apparatus as a video camera.However, other types of processing can also be performed. Also, thepresent invention can be applied to an apparatus having an image signalprocessing function other than the image capturing apparatus.Furthermore, the concept of the present invention can be applied tosignal processing performed on data other than image data, such as audiodata.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image signal processing apparatus comprising: partial image datablock obtaining means for obtaining partial image data blocks bydividing image data corresponding to a predetermined screen having apredetermined number of horizontal and vertical pixels into image dataportions, each having a predetermined number of partial horizontal andvertical pixels, at predetermined positions; packet generating means forgenerating packets, each storing one of the partial image data blocks;and one or more signal processing means for receiving the packets from atransmission path through which the image data is transmitted,performing signal processing in units of partial image data blocksstored in the packets, and outputting the image data in the form ofpackets to the transmission path.
 2. The image signal processingapparatus according to claim 1, wherein the packet generating meansgenerates the packets each including one of the partial image datablocks added with a header, and wherein the header has an area to storeposition information capable of specifying the position of the partialimage data block of the packet in the image data of the original screen.3. The image signal processing apparatus according to claim 2, furthercomprising: image data restoring means for generating signal-processedimage data corresponding to the original image data by specifying thepositions of the partial image data blocks extracted from the packetswith reference to the position information and by combining the partialimage data blocks.
 4. The image signal processing apparatus according toclaim 1, wherein the packet generating means generates the packets eachincluding one of the partial image data blocks added with a header, andwherein the header stores size information capable of indicating thesize of the partial image data block of the packet.
 5. The image signalprocessing apparatus according to claim 1, wherein the packet generatingmeans generates the packets each including one of the partial image datablocks added with a header, and wherein the header has an area to storeparameter specifying information capable of specifying a parameter to beset for predetermined processing performed by the signal processingmeans on the partial image data block of the packet.
 6. The image signalprocessing apparatus according to claim 5, wherein the signal processingmeans determines the signal processing to be performed with reference tothe parameter specifying information stored in the header of the packetbefore performing signal processing on the partial image data blockstored in the packet.
 7. The image signal processing apparatus accordingto claim 1, wherein the packet generating means generates the packetseach including one of the partial image data blocks added with a header,and wherein the header has an area to store signal processing specifyinginformation indicating a plurality of types of the signal processingperformed by the signal processing means on the partial image data blockof the packet, together with the order of the signal processing.
 8. Theimage signal processing apparatus according to claim 7, wherein thesignal processing means determines whether each type of the signalprocessing is to be performed on the partial image data block stored inthe packet with reference to the signal processing specifyinginformation stored in the header of the packet.
 9. The image signalprocessing apparatus according to claim 1, wherein the packet generatingmeans generates the packets each including one of the partial image datablocks added with a header, and wherein the header has an area to storeprogress indicating information indicating progress of each type of thesignal processing performed by the signal processing means on thepartial image data block of the packet.
 10. The image signal processingapparatus according to claim 9, wherein the signal processing meansdetermines whether each type of the signal processing is to be performedon the partial image data block stored in the packet based on whetherthe progress indicating information stored in the input packet indicatesthat the signal processing has been performed.
 11. The image signalprocessing apparatus according to claim 10, wherein the signalprocessing means rewrites the progress indicating information stored inthe packet in accordance with that the signal processing has beenperformed on the partial image data block stored in the packet.
 12. Theimage signal processing apparatus according to claim 1, wherein thepacket generating means generates the packets each including one of thepartial image data blocks added with a header, and wherein the headerhas an area to store processing result information includingpredetermined content that is obtained as a result of the signalprocessing performed by predetermined one the signal processing means onthe partial image data block of the packet.
 13. The image signalprocessing apparatus according to claim 12, wherein the signalprocessing means rewrites the processing result information stored inthe packet in accordance with that the signal processing has beenperformed on the partial image data block stored in the packet.
 14. Theimage signal processing apparatus according to claim 12, wherein thesignal processing means changes and sets a predetermined parameter usedin predetermined signal processing performed by the signal processingmeans based on the processing result information stored in the inputpacket.
 15. The image signal processing apparatus according to claim 1,wherein the packet generating means is capable of generating packets fora moving image based on image data of one screen of the moving image andgenerating packets for a still image based on image data of one screenof the still image, and wherein the signal processing means performs thesignal processing on the partial image data block stored in one of thepackets for the still image during a pause of the signal processingperformed on the partial image data blocks stored in the packets for themoving image.
 16. The image signal processing apparatus according toclaim 1, wherein the packet generating means obtains a predeterminednumber of split partial image data blocks by dividing the partial imagedata block on which predetermined signal processing has been performedby predetermined one of the signal processing means and adds headersgenerated based on the header stored in the packet including theoriginal partial image data block to those split partial image datablocks so as to generate split packets.
 17. The image signal processingapparatus according to claim 1, further comprising: partial image datablock compressing/decompressing means for performing compression whenthe partial image data block stored in the packet is to be transmittedthrough the transmission path and performing decompression correspondingto the compression when the signal processing by the signal processingmeans is to be performed on the partial image data block stored in thepacket.
 18. The image signal processing apparatus according to claim 1,further comprising: partial image data block encrypting/decrypting meansfor performing encrypting when the partial image data block stored inthe packet is to be transmitted through the transmission path andperforming decrypting corresponding to the encrypting when the signalprocessing by the signal processing means is to be performed on thepartial image data block stored in the packet.
 19. An image signalprocessing method comprising: obtaining partial image data blocks bydividing image data corresponding to a predetermined screen having apredetermined number of horizontal and vertical pixels into image dataportions, each having a predetermined number of partial horizontal andvertical pixels, at predetermined positions; generating packets, eachstoring one of the partial image data blocks; and receiving the packetsfrom a transmission path through which the image data is transmitted,performing signal processing in units of partial image data blocksstored in the packets, and outputting the image data in the form ofpackets to the transmission path.
 20. An image signal processingapparatus comprising: a partial image data block obtaining unitconfigured to obtain partial image data blocks by dividing image datacorresponding to a predetermined screen having a predetermined number ofhorizontal and vertical pixels into image data portions, each having apredetermined number of partial horizontal and vertical pixels, atpredetermined positions; a packet generating unit configured to generatepackets, each storing one of the partial image data blocks; and one ormore signal processing units configured to receive the packets from atransmission path through which the image data is transmitted, performsignal processing in units of partial image data blocks stored in thepackets, and output the image data in the form of packets to thetransmission path.